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CLAIMS 

1 • A method comprising: 

providing at least one time-stamped data stream that is to be rendered in 
accordance with data stream timestamps; and 

synchronizing said at least one time-stamped data stream relative to a 
reference by adjusting one or more timestamps of said at least one time-stamped 
data stream. 

2. The method of claim 1, wherein the reference comprises another 
time-stamped data stream. 

3. The method of claim 1, wherein the reference comprises a reference 

clock. 

4. The method of claim 1, wherein the act of synchronizing comprises 
doing so periodically. 

5. The method of claim 1, wherein the act of synchronizing comprises 
determining whether said at least one time-stamped data stream is within a defined 
tolerance and, if not, performing said act of synchronizing. 

6. The method of claim 1, wherein said at least one time-stamped data 
stream comprises a video stream. 
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7. The method of claim 1, wherein said at least one time-stamped data 
stream comprises an audio stream. 

8. The method of claim 1 , wherein said at least one time-stamped data 
stream comprises a video stream and an audio stream. 

9. A method comprising: 

providing a filter graph comprising multiple filters, the filter graph being 
configured to process multiple timestamped data streams for rendering the data 
streams in accordance with data stream timestamps; and 

synchronizing said at least one time-stamped data stream relative to a 
reference by adjusting one or more timestamps of said at least one time-stamped 
data stream. 

10. The method of claim 9, wherein the act of synchronizing comprises 
instructing one or more of the filters to adjust at least one timestamp associated 
with at least one data sample of one or more of the data streams. 

11. The method of claim 10, wherein the act of instructing comprises 
instructing one or more decoder filters to adjust at least one timestamp associated 
with at least one data sample. 

12. The method of claim 9, wherein the reference comprises another 
time-stamped data stream. 
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13. The method of claim 9, wherein the reference comprises a reference 

clock. 

14. The method of claim 9, wherein the act of synchronizing comprises 
doing so periodically. 

15. The method of claim 9, wherein the act of synchronizing comprises 
determining whether said at least one time-stamped data stream is within a defined 
tolerance and, if not, performing said act of synchronizing. 

16. One or more computer-readable media comprising instructions 
which, when executed by one or more processors, cause the one or more 
processors to implement the method of claim 9. 

17. A method comprising: 

providing a filter graph comprising multiple filters, the filter graph being 
configured to process multiple timestamped data streams for rendering the data 
streams in accordance with data stream timestamps; 

providing a synchronization module associated with the filter graph, the 
synchronization module being configured to: 

query individual filters of the filter graph to ascertain input 
timestamp-to-output timestamp mappings, 

compute adjustments that are to be made to output timestamps in 
order to synchronize the data streams, and 
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instruct individual queried filters to adjust their output timestamps in 
accordance with computed adjustments; 

querying individual filters of the filter graph, using the synchronization 
module, to ascertain input timestamp-to-output timestamp mappings; 

computing, based on the ascertained input timestamp-to-output timestamp 
mappings, adjustments that are to be made to output timestamps in order to 
synchronize the data streams; and 

synchronizing the data streams by instructing one or more of the queried 
filters to adjust their output timestamps in accordance with the computed 
adjustments. 

18. The method of claim 17, wherein the act of synchronizing comprises 
doing so periodically. 

19. The method of claim 17, wherein the act of synchronizing comprises 
determining whether the multiple time-stamped data streams are within a defined 
tolerance and, if not, performing said act of synchronizing. 

20. The method of claim 17, wherein the act of querying comprises 
querying individual filters for a current input timestamp and a current output 
timestamp. 
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21. The method of claim 20, wherein the act of computing comprises: 
extrapolating at least one curve characterizing the timestamp mappings for 

at least one of the timestamp mappings to a defined point corresponding to a 
common input timestamp; and 

computing said adjustments based on the extrapolated line(s). 

22. The method of claim 21, wherein the act of extrapolating comprises 
performing a linear extrapolation. 

23. The method of claim 21, wherein the defined point comprises the 
largest value of an input timestamp returned by the act of querying. 

24. The method of claim 21, wherein the act of computing comprises 
computing a skip value by taking the difference between the largest output 
timestamp value at the common input timestamp and the output timestamp value 
for said at least one curve at the common input timestamp. 

25. The method of claim 24, wherein the act of instructing comprises 
instructing one or more filters to jump their output timestamp values by an 
associated skip value at the common input timestamp. 

26. An architecture comprising: 

a filter graph comprising multiple filters, the filter graph being configured 
to process multiple timestamped data streams for rendering the data streams in 
accordance with data stream timestamps; and 



Lee & Hayes, PLLC 



30 



0110021025 MSI - 1 023 US PAT A PP DOC 



1 

2 
3 
4 
5 
6 
7 
8 
9 
10 
11 
12 
13 
14 
15 
16 
17 
18 
19 
20 
21 
22 
23 
24 
25 



a synchronization module associated with the filter graph, the 
synchronization module being configured to: 

query individual filters of the filter graph to ascertain input 
timestamp-to-output timestamp mappings, 

compute adjustments that are to be made to output timestamps in 
order to synchronize the data streams, and 

instruct the queried filters to adjust their output timestamps in 
accordance with its adjustment computations. 

27. The architecture of claim 26, wherein the synchronization module is 
configured to periodically instruct one or more of the queried filters to adjust their 
output timestamps. 

28. The architecture of claim 26, wherein the synchronization module is 
configured to determine whether the multiple time-stamped data streams are 
within a defined tolerance and, if not, at least instruct one or more of the queried 
filters to adjust their output timestamps. 

29. The architecture of claim 26, wherein the synchronization module is 
configured to query individual filters for a current input timestamp and a current 
output timestamp. 
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30. The architecture of claim 29, wherein the synchronization module is 
configured to compute adjustments by: 

extrapolating at least one curve characterizing the timestamp mappings for 
at least one of the timestamp mappings to a defined point corresponding to a 
common input timestamp; and 

computing said adjustments based on the extrapolated line(s). 

31. The architecture of claim 30, wherein the synchronization module is 
configured to linearly extrapolate said at least one curve. 

32. The architecture of claim 30, wherein the defined point comprises 
the largest value of an input timestamp returned by querying the one or more 
filters. 

33. The architecture of claim 30, wherein the synchronization module is 
configured to compute adjustments by taking the difference between the largest 
output timestamp value at the common input timestamp and the output timestamp 
value for said at least one curve at the common input timestamp. 

34. The architecture of claim 33, wherein the synchronization module is 
configured to instruct one or more filters to jump their output timestamp values by 
an associated skip value at the common input timestamp. 
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35. One or more computer-readable media having computer-readable 
instructions thereon which, when executed by one or more processors, cause the 
one or mote processors to: 

query one or more filters configured to process timestamped data streams 
for each filter's input timestamp-to-output timestamp mapping; 
receive responses from the queried filters; 

extrapolate one or more lines characterizing an individual mapping to a 
selected input timestamp value; 

calculate a skip value for one or more of the lines, a skip value representing 
a value by which the output timestamps for a given filter are to be corrected; and 

provide instructions to one or more of the filters to adjust their output 
timestamps in accordance with an associated skip value. 

36. The computer-readable media of claim 35, wherein the instructions 
cause the one or more processors to query the one or more filters for each filter's 
data stream playback rate. 

37. The computer-readable media of claim 35, wherein the instructions 
cause the one or more processors to query one or more decoders filters. 

38. The computer-readable media of claim 35, wherein the instructions 
cause the one or more processors to linearly extrapolate the one or more lines. 
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39. The computer-readable media of claim 35, wherein the selected 
input timestamp value comprises a current input timestamp value for one of the 
filters. 

40. The computer-readable media of claim 35, wherein the selected 
input timestamp value comprises a future input timestamp value. 

41. One or more computer-readable media having computer-readable 
instructions thereon which, when executed by one or more processors, cause the 
one or more processors to: 

define a skip value tolerance that can be used to ascertain when 
synchronization processing of multiple data streams should take place; 

ascertain whether any input timestamp-to-output timestamp mappings 
associated with any of the multiple data streams meet or exceed the skip value 
tolerance; and 

synchronize the multiple data streams if any input timestamp-to-output 
timestamp mappings meet or exceed the skip value tolerance. 

42. The computer-readable media of claim 41, wherein the instructions 
cause the one or more processors to synchronize to a common data stream. 

43. The computer-readable media of claim 41, wherein the instructions 
cause the one or more processors to synchronize to a reference clock. 
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44. The computer-readable media of claim 41, wherein one of the 
multiple data streams comprises a video stream. 

45. The computer-readable media of claim 41, wherein one of the 
multiple data streams comprises an audio stream. 

46. The computer-readable media of claim 41, wherein one of the 
multiple data streams comprises a video stream and another of the multiple data 
streams comprises a video stream. 
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